Demand MemCpy: Overlapping of Computation and Data Transfer for Heterogeneous Computing

Heterogeneous computing relies on collaboration among different types of processors shared data. In systems with discrete accelerators (e.g., GP-GPU), data sharing requires transferring a large amount between CPU and accelerator memories can significantly increase the end-to-end execution time. This paper proposes novel mechanism called Demand MemCpy (DMC) to hide overheads. DMC copies from host memory based demands at page granularity. It utilizes hardware-only fetch requested short latency background pre-copy related pages in advance. Our evaluation shows that reduce time GP-GPU application by 25.4% average overlapping computation transfer not unused pages.